Resistors, Markov Chains & Dynamic Path Planning
نویسنده
چکیده
Dynamic planning involves continuously updating a map by sensing changes in the environment and planning appropriate actions, with all tasks sharing common computational resources. We use harmonic functions for dynamic planning. Analogous representations of harmonic functions as Markov chains and resistor networks are used to develop the notion of escape probability and energy dissipation. These measures are used to indicate convergence (event that permits resources to be devote to nonplanning tasks) more robustly than monitoring maximum or average field changes between iterations. The convergence of the harmonic function is related quadratic-ally to the number of grid elements. An example of an irregular sampling strategy quad tree is developed for harmonic functions, which is complete yet imprecise. Quad trees are not a sufficient sampling strategy for addressing the exponential growth of multiple dimensions and therefore current investigations include other sampling strategies or dimensional parallelization.
منابع مشابه
Expected Duration of Dynamic Markov PERT Networks
Abstract : In this paper , we apply the stochastic dynamic programming to approximate the mean project completion time in dynamic Markov PERT networks. It is assumed that the activity durations are independent random variables with exponential distributions, but some social and economical problems influence the mean of activity durations. It is also assumed that the social problems evolve in ac...
متن کاملDynamic Multi Period Production Planning Problem with Semi Markovian Variable Cost (TECHNICAL NOTE)
This paper develops a method for solving the single product multi-period production-planning problem, in which the production and the inventory costs of each period arc concave and backlogging is not permitted. It is also assumed that the unit variable cost of the production evolves according to a continuous time Markov process. We prove that this production-planning problem can be Stated as a ...
متن کاملStochastic Dynamic Programming with Markov Chains for Optimal Sustainable Control of the Forest Sector with Continuous Cover Forestry
We present a stochastic dynamic programming approach with Markov chains for optimal control of the forest sector. The forest is managed via continuous cover forestry and the complete system is sustainable. Forest industry production, logistic solutions and harvest levels are optimized based on the sequentially revealed states of the markets. Adaptive full system optimization is necessary for co...
متن کاملEstimation of the Entropy Rate of ErgodicMarkov Chains
In this paper an approximation for entropy rate of an ergodic Markov chain via sample path simulation is calculated. Although there is an explicit form of the entropy rate here, the exact computational method is laborious to apply. It is demonstrated that the estimated entropy rate of Markov chain via sample path not only converges to the correct entropy rate but also does it exponential...
متن کاملRobust Planning with (L)RTDP
Stochastic Shortest Path problems (SSPs), a subclass of Markov Decision Problems (MDPs), can be efficiently dealt with using Real-Time Dynamic Programming (RTDP). Yet, MDP models are often uncertain (obtained through statistics or guessing). The usual approach is robust planning: searching for the best policy under the worst model. This paper shows how RTDP can be made robust in the common case...
متن کامل